Extensions to the support vector method

The Support Vector Machine (SVM) is a new technique for solving various function estimation problems. We refer to function estimation as learning, and a technique for estimating the unknown function from data as a learning machine. To construct a learning machine one requires four components: a domain (a learning problem with associated loss function), an induction principle, a set of decision functions, and nally an algorithm that implements all three other components. The present work is devoted to the extension of the Support Vector learning machine. Our objective is to show one can consider using the Support Vector approach in various contexts to solve various problems (in di erent domains, with di erent loss functions, induction principles and sets of decision functions). We argue that the Support Vector approach is a family of related algorithms that share some important common concepts and methodology { the approach is conceptually elegant and well supported by the theory. The material is organized as follows. We start with an introduction to the framework of statistical learning theory (Chapter 1). Chapter 2 describes the Support Vector approach and reviews achievements in the domains of pattern recognition, regression estimation and the solution of linear operator equations. The remainder of the work describes novel extensions to the Support Vector approach. Chapter 3 describes the extension of SVMs to the domain of multiclass pattern recognition, and Chapter 4 the extension of SVMs to the domain of density estimation. Chapter 5 introduces SVMs for pattern recognition which attempt to minimize leave{one{out error. Chapter 6 introduces SVMs with dictionaries of kernels for pattern recognition, regression estimation and density estimation. Chapter 7 introduces a method of transduction for function estimation. Finally, Chapter 8 concludes. 1